Add 'REDISTRIBUTE KEY RANGE' command #789
Conversation
There was a problem hiding this comment.
Great job!
- Please add a description to all methods in balancer.go and coordinator.go
- Please add TODOs to all new methods that you add
- Please add regress tests for redistribute commond (just to check syntax and output)
- Please add as much documentation as possible
- I fount a lot of panics in code. You should assume that a panic will be immediately fatal, for the entire program, or at the very least for the current goroutine. Ask yourself "when this happens, should the application immediately crash?" If yes, use a panic; otherwise, use an error.
| func (rel *DistributedRelation) GetDistributionKeyColumns() []string { | ||
| res := make([]string, len(rel.DistributionKey)) | ||
| for i, col := range rel.DistributionKey { | ||
| // TODO: add hash |
There was a problem hiding this comment.
Don't forget to fix todo and add unittests here
|
|
||
| for _, tt := range []tcase{ | ||
| { | ||
| query: "REDISTRIBUTE KEY RANGE kr1 TO sh2 BATCH SIZE 500", |
There was a problem hiding this comment.
Lets add more corner cases like
- batch size in not specified
- batch size is negative
- what is maximum supported batch size value? Minimum?
There was a problem hiding this comment.
It's not the purpose of this test, it only tests parser, thus will parse any integer constant
| @@ -846,6 +854,12 @@ move_key_range_stmt: | |||
| $$ = &MoveKeyRange{KeyRangeID: $2.KeyRangeID, DestShardID: $4} | |||
| } | |||
|
|
|||
| redistribute_key_range_stmt: | |||
| REDISTRIBUTE key_range_stmt TO any_id BATCH SIZE any_uint | |||
There was a problem hiding this comment.
It would be cool to allow REDISTRIBUTE without batch size, using some default.
| @@ -679,7 +679,7 @@ func (tctx *testContext) stepWaitPostgresqlToRespond(host string) error { | |||
| const trials = 10 | |||
| const timeout = 20 * time.Second | |||
| for i := 0; i < trials; i++ { | |||
| _, err := tctx.queryPostgresql(host, "SELECT 1", struct{}{}) | |||
| _, err := tctx.queryPostgresql(host, "SELECT 1", struct{}{}, postgresqlQueryTimeout) | |||
There was a problem hiding this comment.
postgresqlQueryTimeout -> timeout like in 394 line
There was a problem hiding this comment.
Why do we need it here? Here the default timeout is sufficient
| @@ -97,3 +100,30 @@ func (s *ShardConnect) GetConnStrings() []string { | |||
| } | |||
| return res | |||
| } | |||
|
|
|||
| func (s *ShardConnect) GetConnectionPreferReplica(ctx context.Context) (*pgx.Conn, error) { | |||
| connStrings := s.GetConnStrings() | |||
There was a problem hiding this comment.
Let's at least leave a few tasks here, because ideally we need to make sure that this replica is functional enough.
For example, replication lag is less than value from config or we have metrics from host or maybe smth else.
…MoveTaskGroup is now used inside coordinator only. Its elements are now of tasks.MoveTask tye
EinKrebs
force-pushed
the
redistribute-key-range
branch
from
5391938
to
48f9edb
Compare
No description provided.